180 research outputs found
Boosting Named Entity Recognition with Neural Character Embeddings
Most state-of-the-art named entity recognition (NER) systems rely on
handcrafted features and on the output of other NLP tasks such as
part-of-speech (POS) tagging and text chunking. In this work we propose a
language-independent NER system that uses automatically learned features only.
Our approach is based on the CharWNN deep neural network, which uses word-level
and character-level representations (embeddings) to perform sequential
classification. We perform an extensive number of experiments using two
annotated corpora in two different languages: HAREM I corpus, which contains
texts in Portuguese; and the SPA CoNLL-2002 corpus, which contains texts in
Spanish. Our experimental results shade light on the contribution of neural
character embeddings for NER. Moreover, we demonstrate that the same neural
network which has been successfully applied to POS tagging can also achieve
state-of-the-art results for language-independet NER, using the same
hyperparameters, and without any handcrafted features. For the HAREM I corpus,
CharWNN outperforms the state-of-the-art system by 7.9 points in the F1-score
for the total scenario (ten NE classes), and by 7.2 points in the F1 for the
selective scenario (five NE classes).Comment: 9 page
Classifying Relations by Ranking with Convolutional Neural Networks
Relation classification is an important semantic processing task for which
state-ofthe-art systems still rely on costly handcrafted features. In this work
we tackle the relation classification task using a convolutional neural network
that performs classification by ranking (CR-CNN). We propose a new pairwise
ranking loss function that makes it easy to reduce the impact of artificial
classes. We perform experiments using the the SemEval-2010 Task 8 dataset,
which is designed for the task of classifying the relationship between two
nominals marked in a sentence. Using CRCNN, we outperform the state-of-the-art
for this dataset and achieve a F1 of 84.1 without using any costly handcrafted
features. Additionally, our experimental results show that: (1) our approach is
more effective than CNN followed by a softmax classifier; (2) omitting the
representation of the artificial class Other improves both precision and
recall; and (3) using only word embeddings as input features is enough to
achieve state-of-the-art results if we consider only the text between the two
target nominals.Comment: Accepted as a long paper in the 53rd Annual Meeting of the
Association for Computational Linguistics (ACL 2015
Deep Character-Level Click-Through Rate Prediction for Sponsored Search
Predicting the click-through rate of an advertisement is a critical component
of online advertising platforms. In sponsored search, the click-through rate
estimates the probability that a displayed advertisement is clicked by a user
after she submits a query to the search engine. Commercial search engines
typically rely on machine learning models trained with a large number of
features to make such predictions. This is inevitably requires a lot of
engineering efforts to define, compute, and select the appropriate features. In
this paper, we propose two novel approaches (one working at character level and
the other working at word level) that use deep convolutional neural networks to
predict the click-through rate of a query-advertisement pair. Specially, the
proposed architectures only consider the textual content appearing in a
query-advertisement pair as input, and produce as output a click-through rate
prediction. By comparing the character-level model with the word-level model,
we show that language representation can be learnt from scratch at character
level when trained on enough data. Through extensive experiments using billions
of query-advertisement pairs of a popular commercial search engine, we
demonstrate that both approaches significantly outperform a baseline model
built on well-selected text features and a state-of-the-art word2vec-based
approach. Finally, by combining the predictions of the deep models introduced
in this study with the prediction of the model in production of the same
commercial search engine, we significantly improve the accuracy and the
calibration of the click-through rate prediction of the production system.Comment: SIGIR2017, 10 page
- …